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The Communication Complexity of Achieving 
SK Capacity in a Class of PIN Models 
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Abstract —The communication complexity of achieving secret 
key (SK) capacity in the multiterminal source model of Cslszar 
and Narayan is the minimum rate of public communication 
required to generate a maximal-rate SK. It is well known that 
the minimum rate of communication for omniscience, denoted 
by -Rco> is an upper bound on the communication complexity, 
denoted by Rsk- A source model for which this upper bound is 
tight is called i?sK-maximal. In this paper, we establish a suffi¬ 
cient condition for i?sK-maximality within the class of pairwise 
independent network (PIN) models defined on hypergraphs. This 
allows us to compute i?sK exactly within the class of PIN models 
satisfying this condition. On the other hand, we also provide a 
counterexample that shows that our condition does not in general 
guarantee i?sK-maxlmality for sources beyond PIN models. 

I. Introduction 

Csiszar and Narayan [6] introduced the problem of secret 
key (SK) generation within the multiterminal i.i.d. source 
model. In this model, there are multiple terminals, each of 
which observes a distinct component of a source of correlated 
randomness. The goal is for the terminals to agree on a shared 
SK via communication over an insecure noiseless public 
channel. The SK is to be secured from passive eavesdroppers 
with access to the public channel. The maximum rate of such 
an SK, i.e. the SK capacity, was characterized in [6], and a 
protocol for attaining SK capacity was given, which involved 
communication for omniscience, i.e., all terminals recovering 
the entire information of all the other terminals. However, it 
was pointed out (see remark following Theorem 1 in [6]) 
that omniscience is not always necessary for achieving SK 
capacity. A question that naturally arises is the following (see 
[6, Section VI] and [12, Section V]); what is the minimum rate 
of public communication required to achieve SK capacity? 
We call this minimum rate of public communication the 
communication complexity^ of achieving SK capacity, and 
denote it by i?sK- The protocol from [6] shows that i?sK is 
upper bounded by the minimum rate of public communication 
required for omniscience, denoted by Rco- We refer to sources 
for which this upper bound is tight as RsK-maximal. 

There have been a few attempts at characterizing Rsk- 
In [13, Theorem 3] Tyagi has completely characterized the 
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communication complexity for two terminals in terms of an 
interactive common information, a type of Wyner common 
information [14]. Our previous work [10] involved extension 
of Tyagi’s results to the case of to > 2 terminals. Specifically, 
we gave a lower bound [10, Theorem 2] on the communi¬ 
cation complexity using a multiterminal variant of Tyagi’s 
interactive common information. We were able to evaluate 
this lower bound only in the very special case of a complete 
graph pairwise independent network (PIN) model in which 
we additionally imposed linearity restrictions on the public 
communication allowed [10, Theorem 6]. 

A different approach to analyzing i?sK can be found in 

[3] ,[4]. These follow up on the work in [5], which studied otie- 
shot SK generation (i.e., each component of the source just 
gives out one symbol instead of a sequence of i.i.d. symbols) 
in a hypergraph PIN model, and evaluated the corresponding 
one-shot SK capacity [5, Theorem 6]. This result also used 
communication for omniscience for attaining the one-shot SK 
capacity, but did not address the issue of communication 
complexity. This isssue was addressed in the subsequent work 

[4] , which characterized the communication complexity of 
achieving one-shot SK capacity under linearity restrictions 
on the communication. The characterization was in terms of 
“minimum connected dominating edge sets” of hypergraphs 
[4, Theorem 11]. While the general problem of determining the 
unrestricted communication complexity was left open, it was 
shown that removing the linearity restriction can strictly reduce 
the communication complexity in some cases [4, Theorem 4]. 

The main contribution of this work is the identification of a 
sufficient condition under which a certain class of hypergraph 
PIN models (of which the simple graph PIN models of [12] 
form a subclass) can be shown to be i?sK-inaximal. Thus, 
for this class, we have i?sK = Rco, and the latter can be 
explicitly computed in terms of the parameters of the under¬ 
lying hypergraph. This yields the first explicit computation 
of the (unrestricted) communication complexity i?sK for a 
multiterminal source model with more than two terminals. 
This greatly extends our earlier results from [10], and also, 
in a sense, partially extends the one-shot results of [4] to the 
i.i.d. source sequence model. However, it is also shown via 
a counterexample that our condition does not guarantee i?sK- 
maximality for sources beyond the PIN model. 

The rest of the paper is structured as follows. Section II 
presents the required definitions and notation. Section III 
identifies a class of hypergraph PIN models which are iisK- 
maximal. Section IV shows using a counterexample that the 


results of Section III do not extend to a general multiterminal 
setting. The paper concludes with some remarks in Section V. 

II. Preliminaries 

We will follow the notation and description of [10]. 
Throughout, we use N to denote the set of positive in¬ 
tegers. Consider a set of to > 2 terminals denoted by 
M = {1,2,..., to}. Each terminal i C observes n 
i.i.d. repetitions of a random variable Xi taking values in 
a finite set Xi. The n i.i.d. copies of the random variable 
are denoted by X". The random variables Xi,X 2 , ■ ■ ■ ,Xm 
need not be independent. For any subset A C M, Xa and 
X2 denote the collections of random variables (Xi : i G A) 
and (XP' : i G A), respectively. The terminals communicate 
through a noiseless public channel, any communication sent 
through which is accessible to all terminals and to poten¬ 
tial eavesdroppers as well. An interactive communication is 
a communication f = (fi, f 2 , ■' G fr) with finitely many 
transmissions fj, in which any transmission sent by the ith 
terminal is a deterministic function of Xp and all the previous 
communication, i.e., if terminal i transmits fj, then fj is a 
function only of Xp and /i,..., fj-i- We denote the random 
variable associated with f by F; the support of F is a finite set 
X. The rate of the communication F is defined as ilog|J^|. 
Note that f, F and X implicitly depend on n. 

Definition 1. A common randomness (CR) obtained from 
an interactive communication F is a sequence of random 
variables J^'^\ n £ N, which are functions of Xf^, such 
that for any 0 < e < 1 and for all sufficiently large 
n, there exist X = Ji(Xp,F), i = 1,2, ...,to, satisfying 
Pr[Ji = J 2 = ••• = J™ =/")]> 1 - e. 

When = Xfyi we say that the terminals in Af have 
attained omniscience. The communication F which achieves 
this is called a communication for omniscience. We denote 
the minimum rate of communication for omniscience by Rco. 

Definition 2. A real number R > 0 is an achievable SK 
rate if there exists a CR n £ N, obtained from an 

interactive communication F satisfying, for any e > 0 and for 
all sufficiently large n, I(K^'^^\F) < e and ^H(K^'^'^) > R—e. 
The SK capacity is defined to be the supremum among all 
achievable rates. The CR is called a secret key (SK). 

From now on, we will drop the superscript (n) from both 
and to keep the notation simple. 

The SK capacity can be expressed as [6, Section V], [2] 

liXM) = H(Xm) - max V \bH(Xb\Xbc) (1) 
AeA ' 

BeB 

where B is the set of non-empty, proper subsets of Ai, and 
A = (As ; S £ B) £ A iff As > 0 for all B £ 6 and for 
all i G M, J^B-ieB = 1- It is a fact that \{Xm) > 0 [9, 
Proposition II]. Other equivalent characterizations of I(Xm) 
exist in literature. Theorem 1 of [6] shows that 

(2) 


Theorem 1.1 of [2] and Theorem 2.1 of [1] provides 
yet another characterization of 1(Xm)- Define A('P) = 
fPTT Eagp H{Xa) - HiXM)] ■ Then, 

l{XM)=minA{V) (3) 

the minimum being taken over all partitions V = 
{Ai, A 2 , ■ ■ ■, of A4, of size i > 2. The partition 

{{1}, {2},..., {to}} consisting of to singleton cells will play 
a special role in the later sections of this paper; we call this 
the singleton partition and denote it by <S. The sources where 
S is a minimizer for (3) will henceforth be refered to as 
Type S sources. The following proposition from [11] gives 
us an algorithm to verify whether a source is Type S. For 
any B C A4 with B = {bi, & 2 , • ■ •, ^|b|} denote by Vb the 
partition Vb = {{&i}, {^ 2 }, ■ • •, {h|_B|}i Then we have 

Proposition 1. [11, Proposition 7] For m > 3, let fl = {B C 
[to] : 1 < |i?| < TO — 2}. The singleton partition S is 

(a) a minimizer for I(A[m]) iff A(S) < A{Vb) ^ B gCI; 

(b) the unique minimizer for I(A'[^]) iff A(S) < A(Vb) 

MB G n. 

A better (strongly polynomial-time) algorithm to calculate 
the minimizing partition of (3) has been described in [1]. 
However, Proposition 1 above is more suited for the purposes 
of this paper. 

We are now in a position to make the notion of communi¬ 
cation complexity rigorous. 

Definition 3. A real number R > 0 is said to be an achievable 
rate of interactive communication for maximal-rate SK if for 
all e > 0 and for all sufficiently large n, there exist (i) an 
interactive communication F satisfying — log|A| < R + e, and 
(ii) an SK K obtained from F such that ^H(K) > I(Xm) — e. 

We denote the infimum among all such achievable rates by 
Rsk- 


The proof of Theorem 1 in [6] shows that there exists an 
interactive communication F that enables omniscience at all 
terminals and from which a maximal-rate SK can be obtained. 
Therefore, we have Bsk < Rco < 00 . 


In [10] the communication complexity was lower bounded 
using extensions of proof techniques developed in [13]. The 
lower bound involves a quantity called the interactive common 
information rate, a special case of the Wyner common infor¬ 
mation rate [14] extended to a multiterminal setting. We will 
now define formally what these quantities are. In order to do 
so we need the following extension of the definition of I(XA/t) 
given in (1): for any random variable L, and any n £ N, we 
define 


I(AX,|L)4max 

\a A * 


h(xp^\l)-J2xbH(xp\xp.,l) , 

BeB 

(4) 


where A* C A is the set constituting of optimal A £ A for the 


1{Xm) = H(Xm) - Rco. 





linear program in the definition of 1{Xm) in (1).^ It follows 
from Proposition II in [9] that 1{X^\L) > 0. Also, note that 
liXl,) = nl{XM)- 

Definition 4. A (multiterminal) Wyner common information 
{Ciw) for Xm is o. sequence of finite-valued functions = 

L^'^\X'^) such that 0 as n ^ oo. An 

interactive common information (Cl) for Xm is a Wyner 
common information of the form = {J,F), where F is 
an interactive communication and J is a CR obtained from F. 

Again, we shall drop the superscript (n) from for 
notational simplicity. Wyner common informations L do exist: 
for example, the identity map L = X^ is a Clw- To see 
that CIs (J,F) also exist, observe that J = and a 

communication F enabling omniscience constitute a Clw, and 
hence, a Cl. 

Definition 5. A real number R > 0 is an achievable Clw 
(resp. Cl) rate if there exists a Clw F (resp. a Cl L = {J,F)) 
such that for all e > 0, we have FH{L) < i? + e for all 
sufficiently large n. 

We denote the infimum among all achievable Clw (resp. 
Cl) rates by CIw{Xm) (resp. CI{Xm))- 

To ensure that CI{Xm) < oo, existence of a (J,F) pair 
which is a Clw is needed. Such a pair indeed exists, as the 
proof of [6, Theorem 1] shows that there exists an interactive 
communication F from which a CR J = X^ is obtained, with 
L = (J, F) being a Clw, as discussed after Definition 4. 

The proposition below records the relationships between 
some of the information-theoretic quantities defined so far. 

Proposition 2. [ 10, Proposition 1 ] For any source Xf^, we 
have H(^Xm) ^ CI{Xm) CIw{Xm) ^ 

We conclude this section by stating the lower bound on 
communication complexity as derived in [10]: 

Theorem 3. [10, Theorem 2 ] 


distinct hyperedges in are independent. With this, the 
random variables X", for i G A4, are defined as Xp = • 

e € and i G e). When every e G £ satisfies |e|= t, we 
call % a t-uniform hypergraph. We will show that any Type 
S uniform hypergraph PIN model is i?sK-niaximal. 

Theorem 4. For a Type S PIN model defined on an underlying 
t-uniform hypergraph Ti = {V,£), we have CI{Xm) = 
CIw{Xm) = H{Xm), and hence, Rsk = Rco = 1^1- 

The proof will require two technical lemmas which we state 
below. The first lemma identifies a A G A* when a source is 
Type S. 

Lemma 5. Let the singleton partition S be a minimizer for 
(3). Define X = {Xb ■ B G B) such that Xb = whenever 
\B\= m — 1, and Xb = 0 otherwise. Then X G A*. 

Proof: Observe that A G A. Putting A = A in (1) we have 
H{Xm) - Y.b^b~^bH{Xb\Xbc) = A{S) =l{XMf as 5 
is a minimizer in (3). Thus A is optimal, i.e., A G A*. ■ 

Lemma 6. For any f-uniform hypergraph PIN model and any 
function L of we have: 

m 

Y,I{Xp-L)<tH{L). (5) 

i=l 

The lengthy proof of this lemma is deferred to the Ap¬ 
pendix. We now proceed to prove Theorem 4. 

Proof of Theorem 4: For any Type S source Xm, we 

have 

^ m 

I{XP,\L) > (6) 

i=l 

where (6) follows from (4) and Lemma 5. Now assume 
that Xm arises from a PIN model defined on a f-uniform 
hypergraph TL = (V, £), and consider any function L of Xp^. 
This allows us further simplification of (6): 


Rsk>CI{Xm)-I{Xm)- 


By Proposition 2, the lower bound above is non-negative. 

III. i?sK-MAXIMALITY IN UNIFORM HYPERGRAPH PIN 
MODELS 

This section contains the main result of this work. First 
we will quickly introduce the hypergraph PIN model. The 
model is defined on an underlying hypergraph T-L = (V,f) 
with V = Ai, the set of m terminals of the model, and £ being 
a collection of hyperedges, i.e., subsets of V. For n G N, define 
to be the multi-hypergraph {V,£^'^i), where is the 
multiset of hyperedges formed by taking n copies of each 
hyperedge of R. Associated with each hyperedge e G f is 
a Bernoulli(1/2) random variable the ^gS associated with 

^The maximization earned out in (4) was not originally present in [10]. The 
maximization has been brought in here to make the quantity I(X;vi|L) well 
defined. It can be easily seen that under this modified definition the results 
of [10] are still valid. 


l{XpU\F)>HiXpU)-H{L) 
1 


TO — 1 


[H{Xl,) - H{xp) - nmxp)] 


2=1 
n{t-l)\£\ 




(7) 


n{t-l)\£\ 


TO — 1 TO — 1 


m — 1 \ n J 


Y,I{XP-C)-H{V) 


TO — 1 


Y,I{Xp-C)-tH{C) 


> 


(|£|-lff(L)), 

m — 1 \ n J 


( 8 ) 


the equality (7) using the facts that H{Xp^) = n\£\ and 















1 iJ(X”) = nt\£\, and (8) following from Lemma 6. 

We will now compute C\{Xj^) using Proposition 2. The 
upper bound gives us C\{Xm) < as H{Xm) = \£\- For 
the lower bound, let L be any Clw so that for any e > 0, 
we have ^1{X^ |L) < for all sufficiently large n. The 

bound in (8) thus yields > \£\—e for all sufficiently 

large n. Hence, it follows that C\w{Xm) > \^\- From the 
upper and lower bounds in Proposition 2, we now obtain 

CIw{Xm) = CI{Xm) = H{Xm)- 

Now from Theorem 3 we have i?sK > CI{Xj^) — I{Xj\ 4 ). 
Hence we have 

Rsk > \£\-1{Xm) = H{Xm) - 1{Xm) = i?co, (9) 

where the last equality is from (2). But we also have i?sK < 
Rco, as pointed out in Section II, which proves that i?sK = 
Rco- 

To obtain the exact expression for Rqq, we note that by 
(2) and j3), Rco = H{Xm) - A(5) = ^H[Xm) - 
H{Xi)- This simplifies to the expression stated in 
the theorem using the facts (already mentioned above) that 
H{XM) = \£\^^AYZiH{Xi) = t\£\. m 

We will now show that there indeed exist Type S f-uniform 
hypergraph PIN models. Call Km,t = (V,f) a complete t- 
unifonn hypergraph on m vertices when e C V is contained 
in £ iff |e|= t. Using Proposition 1 we show that complete 
f-uniform hypergraph PIN models are Type S. 

Lemma 7. Complete t-iiniform hypergraph PIN models are 
Type S. 

Proof: Fix a set S C AI with |i3|< m — 2. We calculate 
A{Pb), where Pb is defined as in Proposition 1, and will show 
that A(Pb) > A(5). For Km,t we have, H{Xi) = i^Zi) 
H{Xm) = (™) therefore A(iS) = To evaluate 

A{Pb), note that H{XbZ is the total number of hyperedges 
in £ which contain at least one terminal from Observe 
that if \B\ > f we have H{Xb<=) = (™) — Otherwise, 
we have H{XbZ = (™). 

So first consider \B\ > t. Under this condition we see that 
X{Pb) = ^ H{Xf) + H{XbZ - H{Xm^ 


fm — l\ 

1 - — ( 

^\B\\ 

U-1 J 

1^1 ' 

y t J 


Thus, 

( 10 ) 

1 {m — 1)1 t 
t _{m — f)\ {t — 1)! 

ml 

{t — 2)l {m — t)\ (m — 1) 



1 (m — 1)! / t 

t _{t — 2)l — W — 1 



> 0 


m 
m — 


( 11 ) 

( 12 ) 


where (12) holds as \B\ < m — 2. 

Next consider \B\ < t. Under this condition we have 

A(Pb) = ^ HiXZ + H{Xbc) - H{Xm^ 

— P 
t-1 

Thus, using (10) and (11) we have 


A(Pb) - A(5) = 


/m — l\ 

t-1 

f m\ 

U-1; 

m — 1 

K t) 


_ 1 
t 

> 0 . 


m — 2 
t- 1 


(13) 


Using Proposition 1, (12) and (13), we have the result. ■ 

Remarks. There is in fact a broad class of ordinary graph 
(t = 2) PIN models which are Type S. Corollary 7.2 of 
[11 ] showed that the PIN model on the complete graph on 
m vertices. Km, is Type S. Using Proposition 1, it can be 
easily verified that the Harary graph PIN model (see [8]), 
which contains the complete graph PIN model and the PIN 
model on the m-cycle as subclasses, is Type S. 


IV. Are all Type S sources Psk-maximal? 

Section III showed that Type S PIN models are Psk- 
maximal. A natural question that arises is whether all Type 
S sources are PsK-maximal. The answer turns out to be “No” 
as seen in the following counterexample. 

Example 1. Let W be a Ber(p) rv, for some p G [0,1].' 
Pr[IU = 1] = 1 - Pr[IU = 0] = p. Let Xi,...,Xm be 
rvs that are conditionally independent given W, with 

Pr[A:i = 01|IU = 0] = 1 - Pr[A:i = 00|IU = 0] = 0.5 

and 

Pr[Xi = 11|IU = 1] = 1 - Pr[A:i = 10|IU = 1] = 0.5 

for i = 1,2,... ,m. Denote by h{p) the binary entropy of p. 

It is easy to check that H{Xa) = \ A\ + h{p) for all A C A4, 
and H(Xi\Xj) = 1 for all distinct i,j G M. Therefore, all 
partitions P of M. satisfy A(V) = h(p), and hence, I{Xj^Z) = 
h(p). In particular, Xm defines a Type S source. Furthermore, 
using (2), we have Rco = w. 

We now show that Rsk < Rco- Consider a Slepian-Wolf 
code (see [7, Section 10.3.2]) of rate H{Xi\X 2 ) = 1 for 
terminal 1. All terminals can recover X^ since H(Xi\Xi) = 1 
















for all i £ {2,3, • • • ,m}. Then, using the balanced coloring 
lemma [6, Lemma B3] on Xf, an SK of rate H{Xi) — 
H{Xi\X 2 ) = h{p) can be obtained. Hence, Rsk < 1 < m = 
Rco- 


In fact, there exist non i?sK-maximal sources with S being a 
unique minimizer for (3). To construct such a source we need 
to dehne “clubbing together” of independent multiterminal 
sources on M. Formally for independent sources X'f^ and 
YJ^ dehne the clubbed source as Zf = {Xf, Yf), 
for all z £ JYl. and Ily are dehned to be the sets of 
partitions of Ai which are minimizers of (3) for X^ and 
respectively. We will denote the communication complexity 
(resp. minimum rate of communication for omniscience) for 
the individual sources X"^ and Yf^ by Rskx Rsky (resp. 
Rcox ^nd Rcoy) respectively. The clubbed source satishes the 
following result. 


Proposition 8. Consider two independent multiterminal 
sources X^ and YJ^ and the corresponding clubbed source 
Z^. Then we have 

I{Zm) > nXM) + liYM) (14) 

with equality iff p| Ily 0. 


Proof: Consider any partition V = {Ai, A 2 , ■ ■ ■, A^} of 
Ai. We have 


A{V) = 


e-1 

1 

e-i 


J2h{Za,)-h{Zm) 

■ i 

Y,H{Xa,) - H{Xm) 


Ax(V) 


i-1 


J2h{Ya,)-h{Ym) 


(15) 


Ay(V) 


where (15) follows from the independence of Xf^ and YJ^. 

Thus we have from (15) that minp A('P) > miiiT:? Ax(’P) + 
min'pAy(7^) with equality iff V £ n^fPiny- The result 
follows. ■ 


We conclude the section by constructing a non Rsk- 
maximal source with S being the unique minimizer in (3). 


Example 2. Consider a clubbed source = [Xf ^, ), 

where X'f^ is the source described in Example 1 and YJ^ 
corresponds to the PIN model on the complete graph. So, by 
Lemma 7, we have Ily = {5}. Also, Theorem 4 shows that 
YJ^ is RsK-maximal. 

Since 11^ p| Ily = {5}, Proposition 8 ensures that inde¬ 
pendently running protocols achieving Rskx RsKy’ 

SK capacity of Z^ is attained. Also, (2) and independence 
of X^ and YJ^ show that Rco = Rcox + Rcoy- Therefore, 
Rskx < Rcox (lining Example 1) implies that R^k < Rco- 


V. Concluding Remarks 

The result of Theorem 4 is the hrst exact computation of 
the communication complexity i?sK in a multiterminal source 
model with m > 2 terminals. In general, however, hnding 
computable expressions or bounds for Rsk in a multiterminal 
setting beyond PIN models appears to be a difficult problem. 
On the other hand, a more tractable problem may be that 
of hnding a reasonable characterization of the instances of 
the multiterminal source model which are i?sK-niaximal. This 
seems within reach at least for the class of PIN models. 
For example, one ought to be able to answer the question 
of whether the Type S condition is necessary for (uniform) 
hypergraph PIN models to be i?sK-niaximal. 

References 

[1] C. Chan, A. Al-Bashabsheh, J. Ebrahimid, T. Kaced, T. Liu 
and R.W. Yeung, “Multivariate mutual information inspired by 
secret key agreement,” draft manuscript, Oct. 2014, Available: 
https://www.sites.google.com/site/tieliutamu/research/MMI.pdf. 

[2] C. Chan and L. Zheng, “Mutual dependence for secret key agreement,” 
in Proc. 44th Annual Conference on Information Sciences and Systems 
(CISS), 2010. 

[3] T. A. Courtade and T.R. Halford, “Coded cooperative data exchange for 
a secret key,” Proc. 2014 IEEE Int. Symp. Inf Theory (ISIT 2014), pp. 
776-780. 

[4] T. A. Courtade and T.R. Halford, “Coded cooperative data exchange for 
a secret key,” arxiv:1407.0333vl [cs.lT], 

[5] T.A. Courtade and R.D. Wesel, “Coded cooperative data exchange in 
multihop networks,” IEEE Trans. Inf Theory, vol. 60, no. 2, pp. 1136- 
1158, Feb. 2014. 

[6] I. Csiszar and R Narayan, “Secrecy capacities for multiple terminals,” 
IEEE Trans. Inf. Theory, vol. 50, pp. 3047—3061, Dec. 2004. 

[7] A. El Gamal and Y.H. Kim, Network Information Theory, Cambridge 
University Press, 2011. 

[8] N. Kashyap, M. Mukherjee and Y. Sankai'asubramaniam, “On the secret 
key capacity of the Haraiy graph PIN model,” Proc. 2013 Nat. Conf 
Commun. (NCC 2013), Delhi, India, Feb. 15-17, 2013, pp. 1-5. 

[9] M. Madiman and P. Tetali, “Information inequalities for joint distribu¬ 
tions, with inteipretations and applications,” IEEE Trans. Inf. Theory, 
vol. 56, no. 6, pp. 2699-2713, June 2010. 

[10] M. Mukherjee and N. Kashyap, “On the communication complexity of 
secret key generation in the multiterminal source model,” Proc. 2014 
IEEE Int. Symp. Inf Theory (ISIT 2014), pp. 1151-1155. 

[11] M. Mukherjee, N. Kashyap and Y. Sankarasubramaniam, “Achieving 
SK capacity in the source model: When must all terminals talk?,” Proc. 
2014 IEEE Int. Symp. Inf Theory (ISIT 2014), pp. 1156-1160. 

[12] S. Nitinawarat and P Narayan, “Perfect omniscience, perfect secrecy 
and Steiner tree packing,” IEEE Trans. Inf. Theory, vol. 56, no. 12, pp. 
6490-6500, Dec. 2010. 

[13] H. Tyagi, “Common information and secret key capacity,” IEEE Trans. 
Inf. Theory, vol. 59, no. 9, pp. 5627-5640, Sep. 2013. 

[14] A.D. Wyner, “The common information of two dependent random 
variables,” IEEE Trans. Inf. Theory, vol. IT-21, no. 2, pp. 163-179, 
Mar. 1975. 

[15] A.C. Yao, “Some complexity questions related to distributed computing,” 
in Proc. Ilth Anna. ACM Symp. Theory of Computing (STOC), 1979. 














Appendix: Proof of Lemma 6 

First we state two lemmas which we will require for the 
proof. 

Lemma 9. For independent random variables X,Y and W, 
and any other random variable Z, we have 

I{X-Z\W) < I{X-Z\W,Y). 

Proof: This follows by expanding I{X;Y,Z \ W) in 
two different ways using the chain rule, and noting that 
I{X-Y\W)=Q. ■ 

Lemma 10. For independent random variables X and Y, and 
any other random variable Z, we have 

I{X;Z) + I{Y;Z)<I{X,Y;Z). 

Proof: By Lemma 9, we have I{X]Z) < I{X-, Z\Y), 
and hence, I{X- Z) + I{Y; Z) < I{X; Z\Y) + I{Y; Z) = 
I{X,Y-,Z). m 

We begin the proof of Lemma 6 by arguing that it is enough 
to prove the lemma for the PIN model defined by the com¬ 
plete f-uniform hypergraph Km,t- Consider any hypergraph 
H = (V,f) with |V|= m, and fix a function L of Xf^. Now 
construct a new source Xf^ as follows: first consider the set of 
all f-subsets (i.e., subsets of size t) of V which do not belong 
in £, and call it Associate with each such f-subset e G £^ 
n i.i.d. Ber(l/2) random variables The random variables 
are assumed to be independent of each other and independent 
of those associated with the hyperedges in E. The new source 
Xf^ is defined by Xf = (fXf.{Ij ■. i G G £=}), for all 
i G Ai. Observe that the source X^^ corresponds to the PIN 
model on Km,t- Moreover, we clearly have 

m m 

^/(Xr;L)>^/(Xr;L). 

i=l i=l 

Hence it is enough to show that (5) holds for the PIN model 
on Km,t- 

For the rest of proof we will consider the hypergraph Km,t 
only. We will also use to denote the source described on 
Km,t- We also have /(XJ^;L) = iT(L) from the fact that L 
is a function of . To complete the proof of Lemma 6, we 
will show that the PIN model on Km,t satisfies 

m 

Y, HiC ■■iGe,eG£);L)<t J((C :eG£);L). (16) 

i=l 

For any i G A4, let £i denote the set of hyperedges 
containing i, so that the left-hand side of (16) can be expressed 
® C £i);L). Now, we write £i as a union 
of two disjoint sets £'>i and £^^, i.e., £i = The 

set £^i is the subset of £i containing no terminals from 
{1,2,..., i — 1}. The set £^i is thus the subset of £i containing 
at least one terminal from {l,2,...,i — 1}. Observe that we 
have \£>i\= for 1 <i<m — t + 1 and \£>i\= 0 for 


m — t + 2<i<m. Therefore, 

m 

Ym:--eG£f-,Y) 


2=1 


= 7((er:eGf>i);L) 

m—1+1 - 

2 = 2 - 

+ /((er:eG%);L|(er:eGf^,)) 

m 

+ Y +(C:eGf.);L) 

</((e:eG+i);L) 

771 —1 + 1 / / 

+ E / (C:eG%);L (e:ee|J+, 


2 = 2 

m—1+1 


j<i 


+ E ^((e:eG+ 0 ;L) 

2 = 2 
771 

+ E +(C:eGf.);L) (17) 

2^771 —1 + 2 

m—1+1 

:J((e:eeg);L)+ ^ I {{^ : e G £^Y^^) 


Y im--eG£f;Y) 


( 18 ) 


2=771 —1 + 2 


R 


where (17) follows from Lemma 9. Note that for t = 2, (16) 
follows directly from (18): by virtue of Lemma 10, we have 
Q + R < P, so that the right-hand side (RHS) of (18) is at 
most 2P, as desired. However, the case of f > 2 is not as 
simple and needs further work. 

To achieve the RHS of (16), we require Q + R < (t — 1)P. 
We proceed by defining Q{i) = I ((^" : e G £^i) ;L) for all 
2<i<m — t + 1, and thus, Q = Q(i)- Similarly, 

define R{i) = I ((^" : e G £i) \ Y) for all m — f -f 2 < i < m, 
so that R = 7?(*)- The key ideas are the following: 

1) Expand each Q{i) using the chain rule into conditional 
mutual information terms of the form /(^";L|- • •), and 
further condition them on additional s appropriately. 

2) Allocate these conditional mutual information terms to 
appropriate R{i)s. 

3) Use the chain rule to sum each R{i) and the terms 
allocated to it to obtain P. 

Since the conditional mutual information term L|- • •) can 
only increase upon further conditioning on additional s (by 
Lemma 9), we have Q + R < {t — 1)P as required. 

To proceed, we need to define a total ordering on the set 
£. We represent a hyperedge e as a f-tuple (zi *2 • ■ - it), with 
the ijS, being the terminals which are contained 

in e, ordered according to ii < i 2 <■■■< it- Define a total 









ordering ‘<’ on the set being the lexicographic ordering 

of the f-tuples. Also based on the ordering we index the 
hyperedges of £ as Cj, 1 < j < (7), satisfying < Cj iff 
i < j. As an example. Table I illustrates the indexing of the 
hyperedges in 

TABLE I: Indexing of the hyperedges in 


R{i) is allocated the coming from that Q{k). The table is 
then updated with T{k,j) = 0 to record that the Qe^ from that 
Q{k) is no longer available for allocation. We then increment 
j by 1 and repeat the allocation procedure. Once all s with 
i ^ Cj have been allocated to R{i), we begin the allocation 
procedure for i?(i + l). We formally summarize this allocation 
procedure in Algorithm 1. 


Hyperedge 

Index 

727 

1 

724^ 

2 

727 

3 

734) 

4 

737 

5 

747 

6 

(234) 

7 

(235) 

8 

745^ 

9 

745^ 

10 


Algorithm 1 


To proceed further, using the chain rule we expand each 
Q{i) into a sum of conditional mutual information terms of 
the form Qe = /(f";L|(,fg : e < e,e G £)) as follows; 

Q{i) = m: ■■ e e £^^);L) 

= ^ /(C;L|(eF:e<e,eGf^J) 


< ^ IiC;m2--e<e,ee£)) 

= Qe 


where (19) follows from Lemma 9. Hence, we have Q < 
Eees,. Qe. A total Of [(T-7) ” (7-7) 


i = m — t + 2,j = l. 
while i <m,j < (7) do 

if i ^ Cj then 

k = 2. 

while k < m — t + 1 do 
if T{k,j) = 1 then 

Choose the coming from Q(fc) in (20). 
Add the additional conditioning to make it Qe^^i 
Allocate this term to R{i). 

Tik,j)^0. 

Break. 

end if 

if T{k,j) = 0 && k = m — t + 1 then 

Declare ERROR and halt. 

end if 

1 . 


k k ' 

end while 


(19) 

end if 


j j + 1. 

(20) 



j ^ 1. 

end if 
end while 


{t — Qe terms are generated. Next, each R{i) is 

allocated terms Qe^, 1 < J < (7)’ satisfying i ^ e^. 

This allocation procedure is explained in detail below and is 
also formalized in Algorithm 1. We add a further conditioning 
on each Q^. allocated to R{i) to make it Qe^i^ — I i ^1 (CF • 
e < ej,e € £), : e G £i)). Lemma 9 and the definition of 

Qe.|, ensure that Qe, < Qe^, = 

P. 

We now give a more detailed description of the allo¬ 
cation procedure. Construct a table T with rows indexed 
by i = 2,3,... ,m — t + 1 and the columns indexed by 
j = 1,2,..., (7). This table records the availability (for 
allocation) of a Qe„ from the expansion of Q{i) in (20). 
Initialize the table as follows: T{i,j) = 1 if a Qe^ came from 
Q(i) in (20); else T(i,j) = 0. We carry out the allocation 
procedure on each R{i) in ascending order of i. The procedure 
of allocation is as follows. The idea is to allocate the necessary 
Qej S to R{i) in ascending order of j. Once an i and Cj are 
fixed, we test whether i ^ Cj is satisfied. If not, we increment 
j by 1. If z ^ Cj is satisfied, then the availability of Qe^. from 
Q{k), for all 2 < fc < m — f + 1, is checked using the table 
T. The smallest k which satisfies T{k,j) = 1 is chosen, and 


The flow of Algorithm 1 for is illustrated in Example 
3 further below. We now make the following claims; 

Claim 1. Algorithm 1 never terminates in ERROR. 

Claim 2. Algorithm 1 exhausts all the Qe terms generated in 
(20). 


Claim 1 ensures that each R{i), for all m — f + 2 < z < to, 
is allocated all the Qe„s satisfying z ^ e^. Therefore, using 
Claim 2, we have 


Q + i? — ^ 

m 

^ E 




R{i) + 'y ) Qe 
r-i^ej 

Ri.i) + E 

r-ite. 


it - m 


This completes the proof of Lemma 6, modulo the proofs of 
Claims 1 and 2, which we give below. 

Proof of Claim 1: ERROR is possible only if for some 
TO — f + 2 < z < TO and for some e satisfying z ^ e, all 
the Qe terms generated in (20) have already been allocated. 






















This is impossible as there are always enough Qf,s. To see this, 
suppose e contains t—l—p terminals from {m—t+2, ..., m}, 
i.e., there are p R{i)s requiring an allocation of Qg- Since the 
hypergraph is f-uniform, e must contain p + 1 terminals from 
— f+1}. This implies that the total number of 
QeS generated in (20) is p. Therefore, we clearly have enough 
QeS for all R{i)s. ■ 

Proof of Claim 2: As discussed earlier, the total number 
of Qe terms generated in (20) is (f — Also, the 

total number of Qe terms required by each R{i) is 
Therefore, using Claim 1, the claim follows. ■ 

Example 3. We illustrate how Algorithm 1 proceeds for 
ATs 3 - Denote the hyperedges in £ using 3-tuples, i.e., the 
hyperedge containing terminals 1, 2 and 3 is (123). The 
indexing of £ is illustrated in Table I. So for this case 
we have Q{2) = = 

'^(^(123) ’ ?"l34) ’ ?fl35) ’ ^(234) ’ ?(235) ’ ^)- Thus, (20) takes the 
form 

Q(2) < /(^ri 23 );^) + ■■ e < (124)) 

+ ^(eri25);^l(e:e<(125)) (21) 

Q(3) < /(^ri23);^) +^(^ri34);^l(e : e < (134)) 

+ l^(Cri35);^l(C:e<(135)) 

+ /(Cr235);^l(C:e<(235)) (22) 

Observe that i?(4) and i?(5) require four Qg terms each, and 
a total of eight Qe terms are in fact available from (21) and 
(22). The table T is initialized as follows: 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

2 

1 

1 

1 

0 

0 

0 

0 

0 

0 

0 

3 

1 

0 

0 

1 

1 

0 

I 

1 

0 

0 


We will now illustrate a few of the allocations carried out 
by Algorithm 1. The algorithm begins with i = A and J = 1 
and <5(123) needs to be allocated to i?(4). With k = 2 we see 
that T{k, 1) = 1, and hence we allocate <5(123) coming from 
<5(2) to R{A). The table T is then updated as below. 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

2 

0 

1 

1 

0 

0 

0 

0 

0 

0 

0 

3 

1 

0 

0 

1 

1 

0 

1 

1 

0 

0 


Next we will illustrate the allocation o/<5(i23) to i?(5), i.e., 
1 = 5 and j = 1. The state of the table T just before this step 
is shown below. 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

2 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

3 

1 

0 

0 

1 

0 

0 

1 

0 

0 

0 


Setting k = 2, we see that T{k,l) = 0. So, we move to 
k = 3, for which T{k, 1) = 1. Hence the <5(i23) term coming 
from <5(3) is allocated to R{5), and the table T is updated 
as below. 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

2 

0 

1 

0 

0 

0 

0 

0 

0 

0 

0 

3 

0 

0 

0 

1 

0 

0 

1 

0 

0 

0 


We give one last example of an allocation. Observe that e = 
(234) is the largest (in terms of the ordering on £) hyperedge 
such that Qe needs to be allocated to R(5). We will now 
illustrate this step. This happens when i = 5 and j = 7. The 
updated table T just before this step is shown below. 



1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

2 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

3 

0 

0 

0 

0 

0 

0 

1 

0 

0 

0 


With k = 2, we see that T{k,7) = 0. So set k = 3, and 
note that T(k,7) = 1. So, we allocate to R(f>) the <5(234) 
term contributed by Q(3). Upon updating, the table T now 
has all entries to be 0. Observe that at this point no other 
allocation is required, as the Qe^s for j = 8, 9 and 10 are not 
required by R{5) since terminal 5 is contained in each of e^, 
eg and eig. Thus Algorithm 1 successfully terminates. Finally, 
we rewrite (21) and (22) with underbraces showing the R(i) 
term to which each Qe term was allocated by Algorithm 1. 

Q(2) < /(?(”i23);^-) +^(e(”i24);i-l(er : e < (124)) 

^ ^ ^ ^ ^ ^ 

i?(4) _R(5) 

+ l'(?ri25);^'l(C:e<(125)) (23) 

'-V-' 

R(A) 

Q(3) < /(?ri23);^-) +^(e(”i34);i-l(e : e < (134)) 

'■-V-' '-V-' 

R{b) R(5) 

+ l'(^ri35);^l(C:e<(135)) 

'-V-' 

fl(4) 

+ A^r234);i'l(e:e<(234)) 

'-V-' 

R{5) 

+ I'(^r235);i'l(e:e<(235)) (24) 

'-V-' 

fl(4) 

It can be clearly seen from (23) and (24) that R(i),i = 4, 5, 
have each been allocated with all QeS with i ^ e, and no Qe 
is left unallocated. 





















































































